The trend towards distributed computing, coupled with the pervasiveness of the Internet, has led to a decentralization of resources, such as files and programs, for users and system administrators. As this trend of decentralization continues, user information and data has the potential of being stored on servers and computers remotely located all over the world. The task of monitoring and maintaining such remote resources is physically trying, if not impossible without a method of effective remote access.
To this end, hardware and software solutions have been developed which allow users to access and control computers remotely. Early solutions include software programs that allow text-based control of remotely located computers. An example of this would be a user running a telnet program on a simple Windows-based computer to access files and run programs on a UNIX server. This scheme requires software to run on both the remote server and the user's computer. Furthermore, the telnet programs are normally limited to text.
More advanced software programs have been developed that allow for graphical user interfaces and greater degrees of control. Examples include Windows® XP® remote desktop, and the software program PCAnywhere®. In many of these solutions, a user can control and view a remote computer as if the computer were local, with full control of the mouse and keyboard. However, like the rudimentary telnet schemes, PCAnywhere® relies on software running on both the client computer and the server computer. If either software program fails, the local computer loses control of the remote computer. Thus these solutions are often not robust enough for, among other things, system administration.
Another major drawback of these software solutions is the consumption of processing power (e.g., memory and microprocessor execution time) on the remote computer or server. In addition, once the connection is established, these solutions normally use the remote computer's existing modem or Internet connection to communicate with the local device. Thus, these software solutions consume a substantial portion of the bandwidth available to the server. Both the bandwidth consumption and the power consumption can severely degrade system performance.
In addition, software solutions do not give a system administrator full access to the remote computer at all times. For example, while the computer is rebooting and while the operating system loads, the software program is not running. Therefore, the system administrator does not have access to the server during these periods. This is a major pitfall especially if the system administrator wishes to view or edit BIOS settings or view the server restart (e.g., to perform system upgrades, to debug the system, etc.).
To avoid the aforementioned pitfalls of pure software solutions, system administrators use hardware solutions which are less reliant on the remote server in order to function. For example, keyboard, video, and mouse (“KVM”) switches have been developed that allow a single keyboard, video, and mouse to control multiple computers. The computers are often remotely located from the user or system administrator's computer (i.e., the local computer). These switches route the keyboard and mouse signals of the user computer to one of the remotely located computers chosen by the user. Similarly, the video output of the chosen computer is routed to the attached local monitor. Generally, the user is able to switch to any of a series of remote computers.
Additional hardware solutions include intermediate switches and cables that increase the distance that may separate a user and a remote computer. These solutions can also increase the number of computers a user may control with one keyboard, monitor, and mouse. However this network is generally hampered by a distance limitation.
The KVM switches have advantages over software solutions because they are not reliant upon software running on the remote computer. If a system administrator needs to control and view a computer during “boot up” or to fix a problem with BIOS, the user can accomplish this via a remote keyboard, mouse and monitor linked via a KVM switch. Conversely, this would not be possible with a software solution.
However, most hardware solutions are constrained by distance requirements that limit how far a user workstation can be from a remote computer. Specifically, after a certain distance, signal degradation adversely affects the quality of the video signal transmitted. Therefore, disadvantageously, if a system administrator or user needs access to a computer, the user still has to be within a certain distance of the computer. Although switches and signal repeaters can alleviate the distance requirement to an extent, such solutions are still not feasible for controlling dispersed computing resources (e.g., servers dispersed across a city, state, country, etc.).
A KVM switch whereby the keyboard, video, and mouse signals are sent over standard Internet protocols or telephone connections maybe utilized to circumvent transmission quality over extended distances. This allows any Internet or modem enabled device with a keyboard, video and mouse to control a remote computer regardless of the physical distance between a user computer and a remote device.
However, it has been proven in the art that the creation of such a system is much more difficult to implement than a direct wired KVM switch. In order to send video, keyboard, and monitor signals using a protocol such as those used on the Internet (e.g. TCP/IP, UDP) such analog signals must first be converted to digital signals. The digital signals, in uncompressed form, require a large bandwidth to be transmitted in near real-time. Generally, even high-speed connections such as T1, cable and DSL are incapable of accommodating such bandwidth requirements. Therefore, in order for such a device to be useful in these situations, the analog outputs of conventional monitors must be both converted to a digital signal and compressed.
Video compression ideally exploits redundancies in video signals, both between successive frames of video, and within each individual frame. The transmission of a video signal from a computer monitor output generally has both spatial and interframe redundancies. For example, in a near idle computer, the only change between successive frames of video might be the blinking of a cursor. Even as a user types a document, a majority of the screen does not change over periods of time.
Existing video compression standards are designed for common video applications, and are thus not optimized for KVM switch applications. There thus exists a need in the art for a specialized KVM-specific algorithm capable of taking advantage of temporal redundancy, yet still capable of transmitting changes without a large loss of information.
Further, most forms of video compression known in the art require complicated calculations. For example, the MPEG standards use the discrete cosine transform as part of the compression algorithm. This standard relies on the recognition of “motion” between frames to calculate motion vectors to describe how portions of the image are affected over a period of time. These calculations are complicated and require a combination of expensive hardware, or result in extended transmission periods due to increased computation time.
Finally, many of the existing video compression techniques are lossy (i.e., they reduce the amount of information transmitted in order to reduce bandwidth). Typically, such lossy techniques either reduce the detail of an image or reduce the number of colors. Although reducing colors could be part of an adequate compression solution for computer monitor output, excessive reduction of images may yield a poor video transmission resulting in an illegible video reproduction. For example, if a computer user were attempting to use a word processor, reducing detail could lead to blurry or illegible text.
Several known systems directed to the field of compression and digitization of computer video signals are known, including some systems that operate in an environment of a user computer controlling a remote computer.
For example, a system is known for comparing sub-blocks of an image between successive frames of video and only encoding the differences between the blocks for transmission. In this system, the block-by-block comparisons are completed in the transform domain. Thus the system requires extra computations necessary to compute the transform of the image thereby increasing the time necessary to complete the video compression. In order to obviate the problem and reduce transmission times, this system must require fast and/or complex hardware. The present invention improves upon these time consuming extra computations by completing the block comparisons in the spatial domain. For example, the present invention utilizes a two-level thresholding method to ensure that the block comparisons are effective.
Another system is known that teaches a method of caching previously occurring frames of video to decrease the necessary bandwidth for transmission. The process used by this system compares pixels and retransmits changes between pixels of successive frames of video. Such a system uses a reference memory which may be used to store more than just one previous video frame, allegedly resulting in better cache performance.
Also known in the art of video compression is a method for block-by-block comparison of sequential frames of video, where only changed pixels are retransmitted. This method stores changed pixels in a matrix which is vector-quantized to one of a standard set of matrices. However, this method is likely inadequate for enabling remote control of a computer because it fails to teach methods that take into account noise that maybe introduced into the video through digitization errors.
Another system known in the art is a camera that receives analog video signals and digitizes these signals by implementing a microprocessor that divides the signals into blocks. The blocks of video are then classified and run-length encoded.
A system is also known for digitizing video signals for manipulation and transmission using a method whereby video raster signals from a data processing device are analyzed to determine the information content of the video signal. Although the system uses a method for compressing video, this method relies on analyzing the content of the video and sending that content. Thus, in general, the full graphical interface is not displayed to the user.
Another system is known for high-resolution video conferencing without extreme demands on bandwidth. The system creates a mosaic image by sampling portions of a scene and combining those samples, thus allowing for transmission of video over low bandwidths. Although the system includes a means to reduce the data necessary for transmission, it does not compare frames of video. Further, it does not use lossless compression techniques.
Also known in the art is a system for controlling a remotely located computer. The system uses an interframe block comparison method where pixel values that even slightly change are retransmitted. This leads to the unnecessary retransmission of noisy pixels. The system may also use another method whereby an entire block of pixels is transmitted if a threshold percentage of pixels within the block have changed, which again leads to both unnecessary retransmission of pixels, and/or failure to transmit pixels that should be retransmitted.
For example, if all pixels in the current frame change from black to a dark gray due to noise introduced by the A/D conversion, all pixels will also be retransmitted unnecessarily because the total percentage (i.e. 100% of the pixels) would clearly exceed any predetermined percentage threshold. The system also fails to take into account legitimate changes. For example, an intended change to only a few pixels, e.g., 5 pixels, will be missed if the threshold is set to 6 pixels.
Finally, a system of sending control, status and security functions over a network such as the Internet from one computer to another is also known. In this system, a system administrator can access a KVM switch remotely over the Internet and control the switch. Therefore, in and of itself, this system would not allow a remote computer to be operated over a low-bandwidth connection.
Based on the aforementioned disclosures and related technologies in the art, it is clear that there exists a need for a video compression method designed specifically for remotely monitoring and controlling a computer that is accurate and virtually provided in real-time. Furthermore, there exists a need in the art that allows for platform independent monitoring of computers, even at limited bandwidths provided by standard modem connections. Accordingly, the present invention overcomes the shortcomings of the prior art by recognizing minor changes due to noise, and implementing a more efficient calculation method and with a cache capable of storing previous blocks. Furthermore, the present invention recognizes significant changes (i.e. a pixel changing from black to white due to a cursor). In addition, slight color variations will be smoothed due to the color code and noise reduction methods of the present invention.